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F.VALUA7I0N MODEL 



With the Increasirig numbei- of sophisticated educational 
evaluation models available, any new model should be justified 
before being deyeloped and certainly before its presertation. 
Since the object of this paper is to present Just sucli a new 
nodel , its first step should be to establish the need for the 
model. The major thrust of model building in educational evalu- 
ation has been toward general models. Certatnly those of Scrlven 
(1967), Stake (1967), Provus (1971), Stuffl ebeani, et al ■ (1971) 
and Hammond (1 967) He at this 1 evel . Considerable^ attention 
has been paid to the evaluation of educational products. Grobman 
'(1968) has presented an account of the procedures used in the 
BSCS project. Scrlven has recently proposed a coraprehensi ve 
product evaluation profile. Of the few eyaluations published, 
most seem to be of educational products if one defires that 
broadly to include course v/orte, Of the more specific evaluation 
activities, evaluation of students has received considerable 
attention, perhaps most comprehensively by Bloom, et al . (1971). 
Although receiving less attention, evaluation of faculty has also 
received considerable attention and an extanslve literature on that 
subject exists. (Miller, 1972) 

Between the general niodels of the evaluation process and the 
specific models and methodologies for evaluating people, products 
and processes, there seems to be a gap. Specifically, this gap 
lies at the level of the educatioral organization. The general 
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nradels do net define for an organization the specific areas that 
it should evaluate, At the same times the specffic procedures 
for evaluation of studentSi faculty, products, etc. tell how to 
evaluate but not v/hen or to w?hat end. There are two existing 
responses to the problem of evaluation of iducational orgamiiatfons. 
The first is the accredi tation/sel f--stud^ fncdeT. This fnodel 1s 
widely used and has a number of advantages, Hoiyeverg as Wortheri 
and Sanders (1973) have pointed out,, the accreditation modeVs 
objectivity and empirical basis are questionable j its attention 
to process 1s not balanced by eqiual attention to the cortsequerices 
of educatloinj and its replicabnity is questionable. It also 
suffers froni being in large measure ^ an eKternal ly imposed process. 
The coniCluslons resulting from it are frequeiitl/ resisted vn'thin 
the educatlgnal organization itself. The other existing response 
is instl tutlonal research. Offices of institutional research have 
demor?strated their usefulness In the uhiversit^^ context (Dressel , 
1972) J but are largely/ unlcnown belovv that lev^el. Both the accredi- 
tation teams and Institutional research offices^ typical ly operate 
v/tthout detailed models. Ihus^ there Is a need for a model for 
evaluating educational organisations. 

In contrast to education ^ a n^urter cf writers 1n the business 
managennent field have explored the processes inv^olved 1n evaluating 
organizational activitiest e,g. Drucker (196^* 1974)» Li kert (1967, 
1960) and Huffible (1958) . 

These models* use and influence in the business coranunlty can 
attest to their effectiveness in that context* Howev^erj the transi- 

4 

2 

ERIC 



t1on from a business to an iducational context has frequently 
been difficult due to the differences betv/een the profit and 
corijnunity service orientation* Profn the standpoint of evaluating 
an organization s the business cortteKt has the impDrtarit advantage 
of having a widely accepted sta!idard,j profit or return on invest^ 
ment^ v/ith v^hlch to work. There is ro equivalent standard for 
judging the perforniance of non-profit and public educational 
organizations. As it turns out^ business organizations , especially 
large ones> must Increasingly confront this same probleni of 
stanctards. Their Internal operations do not, for the most part, 
mke a direct, unarnbiguous con^tri bution to the ultimate goal. 
These rnodel s are various attempta to confront the problem of 
arriving at standards by which organiiations can be judged, ThuSj 
they have much to contribute to a simnar problem in educational 
organ1zat1onSi The purpose of this paper 1s to draw from the 
fields of both educational evaluation and business mnagement to 
produce a model specifying the necessary processes for evaluating 
an ediicatlonal organization. 

ModQl E valuation 

One of the hopeful things about the educational evaluation 
community is its tendency to apply its Ideas to own products, I 
hope that you will share in thi> practice by evaluating this model 
both as you read about it and after you finish, Worthan and Sanders 
(1973) have compned a table critiqiiing most of the proniinent edu- 
cational evaluation rnodels, Figure 1 Is adapted from their table 
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and lists both desirable and undeslrabie aspects of the educational 
evaluatfon model itself, I suggest jou consider to what degree 
the model about to be prasented possesses these characteristics . 
I will return to the question of evaluating this model at the 
end of the paper. 

Model Assufuptlons 

The key to anj model lies In the assumptions upon v/hich it 
is based. Sometimes model assymptions are not well defiried. But 
clearly different assumptions will result in different niodels^ aad 
a model v/ill be far less useful to someone v/ho does not shiare its 
assuniptions than to people who do. As for this model ^ it is based 
*on four main assumptions about organizations, their purposes and 
functioning^ 

1, An organizations pLirpose and its results He not in Itself ^ 
but in the community in which It eKists. In one sense^ this Is 
obvious, In any society with a division of labor^ people and 
organizations must exchange and thus are Interdependent, But sofne 
of the inipli cations of this fact are not obvious. The most im- 
portant is that one cannot tell froiii a perspective internal to the 
organization what services are useful to the outside conniunity. 
Without knovfing how the services or products of the organization 
are valued ^ it 1s impossible to tell which activities and costs 
within the organization are productiye and v/hich are v^aste. All 
one can tell from within the organisation is that costs are being 
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generated. An Important error to be avoided is equating these 
costs generated by the organization with the benefits received 
in the corrmunity. In almost all organizations , effort Is w^asted 
and thus produces no benefits in the comniunity* 

2, Results are produced by a combination of repeating past 
processes and by creating new ones. The repeating of past processes 
is the v/ay most things are done most of the time and thus it pre- 
dominates- Unfortunately 3 pure repetition can only result in 
things getting worse as new situations make old solutions Uss 
applicable^ Creating nev/ processes comprises a far smaller amoiirt 
of the 'time spent in organizations ^ but at the same time It is the 
only v/ay in which mankind can hold its own or make progress, 

3. The progress value of new ideas 1s used up as the ideas 

are Implemented* In terms of the idea Itself, Its ability to create 
progress is high before it is applied. Frequently It retairs signi- 
ficant potential for new progress as it Is used in new situations or 
combined v/ith other ideas. But eventual ly all useful applications 
are found and exhausted, The idea may even become counter-productive 
as situations change but its application continues even though it has 
becorne Inappropriate* The same process occurs 1n terms of organizations . 
Initially the organization which originated the idea makes use of it 
and is nKDre productive than other organizations. But as the other 
organizations see the ideals usefulnessi they adopt It and gain 
productivity. Again * the originating organization can eventually 
become less productive if it holds on and continues to apply the 
idea after it has become Inappropriate, 
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4* What is distinctive about organl2ations are their people 
and the InterreUtionships between those people* Facnities and 
equipment can be duplicated, Even knowledge is a resource widely 
available in the culture* Successful organizations are frequently 
CDpied as to their exterior configurations, but the successful 
organizations distinctiveness persists because it lies in the 
unique relationships betv^een the people in the organization. 

This FDodel 1s geared to evaluating organizational effective- 
ness In terms of these assumptions. It 1s concerned with deter- 
mining the needs of the conifnuni ty and judging the results of the 
organization In terms of those needs* It looks at the organization 
in terms of its ability to generate new solutions and to keep that 
process going* Finally ^ It defines the crganlzatlon and Its 
processes prlinarlly in terms of the organization's people , their 
knowledges and Interrel atloiishi ps* 

The Model 

The model envisions the organizations Interacting with the 
community across tv/o mjQv Interfaces, as Illustrated in Figure 2. 
The first is the resource interface. The organization drav/s 
resources from the cormunlty. The most important resoiirces that 
educational organizations receive, are the time of the student, 
teachers, and administrators end buildings in which they are housed. 

, The 

second interface betv/een the conmunity and the educational organiza- 
tion is the point at v/hich the educational orgarjss tion* s products 
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or output is returned to the coiratiutiity ae students leave the school* 
Resources have been utilized to produce changes* typically in the 
behavior of the students. 

The goal is for the organizations Interaction with the 
iiiputs to transform them so that thej/ are more valuable (in 
wfii ch form they are then called the outputs). In general ^ that 
goal 13 easily met and assessed. The more demanding goal (and 
correspondingly harder to assess) Is i^hether that Increase in 
value 1s a greater one than could have been achieved elsewhere. 
In other words i v/e have long had the knowledge sufficient to 
effect some Increase in value through education. There are in- 
nuniirable v/ays to teach reading for in stance j most of which work 
in some degree. The question that needs to be answered is whether 
the methods used by the organl2ation are better^ that 1s 5 produce 
more value for less costs than alternative ways. 

One way to approach this cost=benef1t analysis is to break 
its consideration into two parts^ the inputs and the outputs. In 
both cases there will be standard usages, For instance , students 
plus teachers plus books plus a classroom equals the standard v/ay 
of producing better readers. There are fairly clear definitions 
of each component as v^ell as the eKpected change in the students* 
If our goal Is to produce more value for less resources 5 we can 
attempt to find resources or Inputs that are not standard and 
cost less in an attempt to accomplish the same goal for less. 
Ors %'e cm turn our standard resoiirces to producing outputs that 
are not standard and are more valuable. Or we can attempt to 
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dD both. In the case of the readers, we might find that student 
tutors using programmed texts produce the same result at far less 
cost. Alternatively^ we might find that instead of taaching 
reading, we might teach auto repair;, and the. students .learn: about. : 
as much reading and have a salable skill. Finally^ we mght use 
mechanics, student tutors and progranimed tests to teach auto repair 
and find that we gain at both ends of the- process. We have used 
less resources to produce a more valuable end result. If we then 
pull together the results from the various programs , we have not 
only a clear assessment of the organization's worth (i*e. v^hat 
v^alues it produces over v^hat could be produced with the same 
resources in alternative uses)^ but also a clear guide to action. 

To produce such an evaluations we now must return to the 
community to assess its needs and resources* In business, the 
process of identifying needs 1s called a marketing analysis or 
survey, 

< V- _ In education a very slmnar basic idea is called 
needs analysis or assessment. There are a number of systematic 
nwdels in this area as v/ell . (Hammond^ 1967; Grotelueshchen and 
GoDler, 1972 and Poham, 1 972), See also Bailey and El 11s <1974) 
for a brief review of the literature on this subject. Most methods 
aim at existing needs that can be satisfied by existing [nethods. 
At any given time, most of those needs will have been met. The 
best that can be expected is that some small improvements for iso- 
lated groups of people will be possible. The real latge scale 
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improvements that are possible are going to be in ideiitlf^ing needs 
that people are not aware of or are resigned to living without or 
which finding new methods to meet old needs. The point is that 
this 1s not slfnply a systematic process requiring little Imagination* 
Heuristic methods that look for the unexpected are essentlaU 
There must be the addition of Imagination and vision to make it a 
process of great value. 

From the needs analysis we get the information on needs from 
which we can begin our analysis of the organizations products. 
Again the processes involved have been developed eKtensively by 
others. Gottham and Clasen (1972) have taken Tyler's paradigm 
and given detaned steps as to hov/ to utilize 1t, Grogham (1968) 
*gives a lengthy account of how to evaluate curriculum materials. 
A very useful tool to summarize and integrate much of the 1nforma = 
tion is the Product Evaluation Profile developed by Scri ven .(1974) . , ^ 
Drucker (1954) also has a methodology derived from business management 
studies that can be usefully applied* 

However^ an organization 1s not simply its envlroninent nor its 
products. The organization exists to transform the inputs into 
more valuable outputs* There are two aspects of the organization 
that are pertinent here. The first is that the process generates 
costs* The second is that the only thing that can really be distinc- 
tive In any organization are its people and their interrelationships. 

Lets break this people distinctiveness into two parts, the 
knowledge of the people and their interrelationships. The first 
part of this pair is the most f undarmntal , If the knov^ledges and 
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skillsp broadlj defined, of the people in an organization are 
inappropriate to its task, no amount of money, materials or 
organizational structLire can make it productive* There are only 
two available alternatives* The first 1s to create the appropriate 
knowledges and skills for the task at hand. The second Is to find 
a task that the knowledges and skins fit. Evaluating an organiza- 
tion on this dimension Is not a task that thas been extensively 
pursued* And ^et it is a critically important task. Since almost 
inevitably there is a tendency for people within an organization to 
let their skills to become dated or Irrelevant. 

dy definition of v/hat It means to be an organization, the 
fruits of the knowledges and skills of the people within it are 
realized through an organizational structure. Organizations e^lst 
to enable an organized group of individuals to do things that the 
same individuals could not do individually. As such, they date back 
far past the origin of humankind. Howeveri as numerous studies have 
shown (Chandlerp 1962^ Penrose ^ 1959) some organizations are less 
effective than an individual and^ more commonlyj many organizations 
operate at a far lower level of productivity than is optimally 
possible* ThuSj there are rare instances where an organization actually 
decreases a person s.^effectlveness^ but, more usuanyj organizations 
simply do not gain the high degree 'of effectiveness that Is possible* 
An appropriate organizational structure 1s the critical element in 
determining whether the knowledges and skills of the people can be 
effectively appHsd to the task. Of course^ the ability to create 
and maintain an effective organizational structure is a set of 
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knowledges and skills that some Individual (s) w1th1n the organiza- 
tion must have, Likert (1967) has suggested that a system of human 
assit accounting be created to measure the resources tied up in" 
the knowledges and skills of the people in the organization as well 
as the effort invested In creating a good organizational strUQture* 
An enormous Investment of time and monty will have beerimade to 
bring together people with the right skills and than create an 
organization to ma5<1m1ze their effectiveness. That investment is 
not recorded on an^^ books of organizations either public or private 
Since It Is not accounted for^ it is often forgotteni Since an 
organization's results are alv/ays outside the organization ^ it can 
never tell whether its actions are producing value or mere waste 
froiD an internal analysis. All it can tell is that its actions 
cost. There is inevitably some waste and frequently enormous 
waste in any process* While. some useful attacks can be made on 
this problem from a purely internal analysis 5 the most important 
analysis will be from an outside perspective. Drucker (1964) has 
a useful system for an overall analysis and categorization of costs 
Beyond his general scheniQ there is the v^hole field of aceounting 
with its subdivision as well as various branches: of industrial 
engineering and operations research. But unless there is a clear 
understanding that costs results, none of these tools v/ill be used. 
Unfortunately most educational organizations have not or are only 
beginning to vaguely realize this fact. 

Once the data has been collected, the problem of integrating 
the results remains. This integration can be approached with two 
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dlfferefit intentions. The first Is with the goal of judging the 
organization I t^picanj on a good-bad or go-iiD go dimension* This 
type of JudgiTiental approacli has beer used most frequently on edu- 
cational products In the past for the very i^cellent reason that 
there are frequently jiidgmantal questions such as whather to adopt 
a particular product or not* In educatiDns about the only equiva- 
lent judginent of organizations is the accreditation process* The 
difficulty here 1s in specifying what Is important* How much 
success in one area trades off for lack of success in another area? 
It is important that the process of specifying standards occur* The 
model suggests areas in which standards must be set, 

The second approach to Integratirig the results of the studies 
>s from the perspective of the decision-maker. This perspective 
probably has a far greater utility since decision-niakers have the 
responsibn Iti es for improving thiir organizations on a continuing 
basis, Integratirig the results from this perspective is a tv/o step 
prqcess. The first is identifying the discrepancies that have been 
isolated. The sicond Is identifying the appropriate action to be 
taken 1r response to these findings. 

There are four general types of actions that; can be. taken based 

on the Information generated by this model of evaluation. The most 

important type of action is the allocation of resources* The 

easiest decision Is to congratulate successful operations but take 

their key people and resources to throw in support of the failing 

operations 1n an attempt to salvage them* The fallacy of such an 

.1 

approach is relatively cleart The correct approach is to support 
success ard abandQn waste as quickly as 1s feasibla, But whatever 
is done that decision v^ill probably be the most ifn^ortant result 
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of the evaluatloii. The other types of actions Involve creating 
something new. Hev products or programs can be divaloped, New 
knowledge can bi acquired/ Mew organisational practicas can be 
developed. Each of these actions can be taken in responsf to the 
discrepancies that are discovered In the aval uati on process. 

What tjpes of discrepancies are typically discovered? The 
first type of discrepancy Is between the products or programs of 
the organization and the needs in the conmiinity. This type of 
discrepancy can point out any ont of three actions and most likely 
all three. First It can point put new products or programs that 
need to be developed. It can also point out products or programs 
that are successful but are not being suppcrted sufficiently by 
the organization to take advantage of their success, Almost 
iiievltably there will be failures that need to be terminated or 
drastically modified as soon as possible. 

The second type of discrepancy is betw/een the knov/1 edges of 
the people within the organization and the knowledges required by 
tlia situation It points to the nevv knowledges that fnust be added 
to make the organization be effective. Alternatively, it can point 
to new directions of developinent that are more appropriate to the 
e5(i$t1ng knowledges. Ihe Idea of additig knov/ledge to existing 

knowledges In order to be effective at what the organization is 
doing now seems the easiest and most sensible. Howeveri the diffi- 
culty in changing what the people in the organization are good at 
should not be underestimated. A change of direction toward the 
development of products or programs that use the existing knowledges 
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my easily b€ the best direction aspecially if tfie kiiQwIedgas have 
changed to a marked degree. 

One v/ould expect that a discrepancy discovered In the analysts 
of the organizational structure woul d lead to nev* organizational 
practices. Ihat Is the most coprKin result^ but It is not the oaly 
possiblHt^/. Problems 1r the organisational structure maj? pQiTit 
to the need for new knowl edges t especlaVly tn the handling of people. 
The>' may also point to the need for new developments. The orgimiza- 
tional structure maj not be geared to doing well some of the things 
that are done now^ but is capable of biing aKtreTOly effective in 
other areas* Again, it is often more effective mt to tear apart 
the organiaalion simply to be able to continue doing what was done 
before* As is true of krowledgeSs organisational structures are not 
easi^ to change for the better. If they can he adapted to a more 
appropriate task, that may be the wrm useful solution. 

The final general category of discrepancies is that of costs. 
The most common discrepancy in this area is the Identification of 
wastefiil activities. The appropriate decision is cut theni out* The 
effectiveness of this decision depends greatly m the comnitttn]€iit 
to it. Beyond siinply eliriinating v/aste , discrepancies in the cost 
data can call for nev/ developr^ents ^ knowledyes and organiaatioral 
structures. Nev^ developTOnts can be useful v/hen existing processes 
are infle)c1blej excessively costly, or v/here other products can 
be spun off fr^m thm to take advantage of mrk that Is in\^olved in 
their production. New knov/1 edges may be required in areas that were 
previously marginal but ncv/ are major activities, Nev/ organizational 
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structures may cut costs. 

Thus the integration of the results from the standpoint of the 
dec1siDn-maker is going to be 1n terms of actions. While it Is not 
a requirements ths evaluation is an obvious Juniping-off point for a 
major planning process, The data aind the implications of the evalua- 
tion v/lll provide the factual conteiit necessari^* eyaliJatlon 
process Is likely to have gone an important way tQward. ganerating 
the interest and establishing the necessary attitude to support the 
planning process* 

CQNCLIJSION 

By wa3^ of summary , let me eyaluate the model that I have 
p'roposed in terms of the standards contained in Figure U IJaturany 
I win rate it as an advocate. It possesses all of th^ desirable 
characteristics- Froin an overall standpoint it 1s capable of both 
formative and summatlve evaluation. If fully carried out, it produces 
an' eval uatloTi of the ofgani nation, the same time, that evaluation 
is a guide to action for Improving the organtzation 3 its products and 
processes* The model 1s general izabl e to all types of educational 
organizations. It is not: tied to a-spec1fi€; type^ or s4z& of orga 
tion. The model does directly assess the worth of the educational 
organization, It provides for not only collecting data but for 
integrating the data into judgements. Finally^ 1t is whoHstlc, It 
provides for including not onlj all aspects of tlie organization but 
for the commiinity in which the educational organiEatlcn operates. 

In terms of undesirable characteristics^ there is unquestionably 
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a problem with bcth the clarity of the prestntatlcn arid of the 
concepts themselves. The model is ,an attenipt to Integrate matirial 
from two distinct fields; each with its own distinctiv^e ccricepts, 
Imyittbly there will be misunderstandings. This model also suffers 
from not having been entirely converted into i working set of methods. 
It is onli^ In the initial staLges of field testing* ThuSi U requires 
further refinement arid clarlficattom. This paper fs a st€p In that 
direction. It is less vulnerable to criticise on the other undesir- 
able characteristf cs. The wodel Is concerned v/lth values from Its 
first step^ looking at the raids of the community, to its lasts in- 
tagrating the data into judgements of worth. Nor can it ba cr-itictzed 
for not evaluatirig, It evaluates products and sarvices of the organi- 
iation, its processes and organisational structure, tlie krowledge of 
its: people, In ternis of their ability to meet the needs of the 
communit^^* Finally^ it nalrtains a balanced focus on process and 
cutconie. It relates the organizations processes to Its prodLicts 
and judges thenn as an Integrated whole. 

If this fnodel does indeed possess all the virtues dust ascribed 
to its the next step is to produce a methodclogy v/l th which to Implenient 
It, Due to considerations of length , this v^lll not be attempted in this 
paper. Hov/evers a prellniinery version is currently being field tested, 
The various methodologies inentloned in the paper can be used on an 
interini basis, especially valuable are the ones contained in Drucker 
(1964) and Scrlven (1974). 
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Good ChairacteristlcS i 

Xi EncprpPTates both foMatlv^e and suTO^att^e 
EvaLvatlon* 

2* Genexallzable to many evaLuatlQn situations. 
3» Directly assesses %^orth of evaLuated acttvity 
4. P^ollstlc, 

Bad Chaj'_aj^tg_T4^tic_s_ f 

L. Lacks clarity and lias overlepplng concepta, 
2t Places little eniphasls on valiifts* 

3, Not e^faluatlon - descriptive or plaiining 
liiforiQatlon, 

4, Focuses on procesi or oiitcotne to e^cluslofi 
of D tiler* 



Figure 1 , E\faluatloti model characteristics , ( Adapted 

troTij I^oTtlien, B* R,, & Sanders, J, R. Educational 
CVjaluatior : Jheory and Jra^_Ll_c:_u , l^orthingtoris 
Ohioi Cliarles A. Jones PijblishJng^ 1973.) 
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Pigtire 2, ConmuriiCy - OrgaTiLzatioa Interfaces, 
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